Asymptotically Exact Error Analysis for the Generalized $\ell_2^2$-LASSO

Given an unknown signal x0 ∈ R and linear noisy measurements y = Ax0 + σv ∈ R, the generalized `2-LASSO solves x̂ := argminx 12‖y −Ax‖ 2 2 + σλf(x). Here, f is a convex regularization function (e.g. `1-norm, nuclearnorm) aiming to promote the structure of x0 (e.g. sparse, lowrank), and, λ ≥ 0 is the regularizer parameter. A related optimization problem, though not as popular or well-known, is often referred to as the generalized `2-LASSO and takes the form x̂ := argminx ‖y−Ax‖2+λf(x), and has been analyzed in [1]. [1] further made conjectures about the performance of the generalized `2-LASSO. This paper establishes these conjectures rigorously. We measure performance with the normalized squared error NSE(σ) := ‖x̂− x0‖2/σ. Assuming the entries of A and v be i.i.d. standard normal, we precisely characterize the “asymptotic NSE” aNSE := limσ→0 NSE(σ) when the problem dimensions m,n tend to infinity in a proportional manner. The role of λ, f and x0 is explicitly captured in the derived expression via means of a single geometric quantity, the Gaussian distance to the subdifferential. We conjecture that aNSE = supσ>0 NSE(σ). We include detailed discussions on the interpretation of our result, make connections to relevant literature and perform computational experiments that validate our theoretical findings.